Skip to content

feat: distribute fmtkit as a single self-contained binary (Homebrew + GitHub Releases)#48

Merged
gocanto merged 8 commits into
mainfrom
feat/single-binary-distribution
Jul 16, 2026
Merged

feat: distribute fmtkit as a single self-contained binary (Homebrew + GitHub Releases)#48
gocanto merged 8 commits into
mainfrom
feat/single-binary-distribution

Conversation

@gocanto

@gocanto gocanto commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Delivers the distribution goal that #46's root-module collapse was the prerequisite for: one self-contained fmtkit binary per platform, downloadable from GitHub Releases and installable via Homebrew (brew tap oullin/fmtkit && brew install --cask fmtkit) — no Node.js, no Docker, no npm at runtime.

How it works

fmtkit (Go binary, per platform, ~90-100MB / ~35-49MB tar.gz)
├─ go:embed fmtkit-ts-sidecar   bun-compiled bundle of the devx TS pipeline
├─ go:embed oxc-parser.node     napi bindings, loaded through
├─ go:embed oxfmt.node          NAPI_RS_NATIVE_LIBRARY_PATH
├─ go:embed oxlint.node
└─ go:embed .oxfmtrc.json / .oxlintrc.json

First run extracts the toolchain to the user cache (~/Library/Caches/fmtkit/<version> / $XDG_CACHE_HOME/fmtkit/<version>); after that everything is spawned from there. Go formatting runs in-process via the existing internal/cli runner, and file collection reuses internal/sourcefiles instead of the fmtkit-sources child process.

  • packages/devx/scripts/sidecar.ts — multiplexes pipeline/oxfmt/oxlint through one Bun executable so the runtime ships once.
  • infra/scripts/release/stage-ts-assets.sh — cross-compiles the sidecar (bun build --compile --target ...) and fetches the platform napi bindings into gitignored infra/bin/<os>_<arch>/ directories; versions pinned by packages/devx/package.json.
  • embedded_sidecar_*.go (module root, where go:embed can reach infra/bin/) — per-platform embeds behind the fmtkit_sidecar build tag (dev builds stay light).
  • packages/driver/internal/tsruntime — cache extraction with atomic rename, tool spawning with the existing env-override contract (FMTKIT_SUPPORT_DIR, OXFMT_BIN, OXLINT_BIN, ...); project-local .oxfmtrc.* / .oxlintrc* files still win over the bundled configs.
  • packages/driver/internal/orchestrator — Go port of infra/bin/fmtkit: sectioned/colorized progress, live-streamed prettified tool logs plus the condensed summaries; --quiet restores summary-only output.
  • packages/driver/cmd/fmtkit — the distributed CLI: format, format-all, ts, lint, go <check|format|sources|...>, check, version, help. format/format-all take --ts (TS/Vue format + lint only) and --go (Go formatting only); no flags runs everything.
  • .goreleaser.yaml + publish-release.yml binaries job — archives + checksums attached to the existing release, Homebrew cask pushed to oullin/homebrew-fmtkit.
  • tests.yml binary job — goreleaser check + an end-to-end smoke test of the compiled binary on a scratch project.

Docker images and the infra/bin/* entrypoints are untouched; a follow-up can swap Dockerfile.full onto the Go orchestrator.

Action needed before the first release

Create a fine-grained PAT with Contents: read/write on oullin/homebrew-fmtkit and save it as the HOMEBREW_TAP_TOKEN repo secret — GITHUB_TOKEN cannot push cross-repo. Until it exists, the binaries job will fail at the cask-push step.

Validation

  • go test ./..., golangci-lint run (driver), pnpm test + pnpm typecheck (devx) all green.
  • goreleaser release --snapshot --clean builds all four platforms; the darwin-arm64 dist binary formats a scratch TS+Go project end to end, extracting to the versioned cache.
  • ./infra/scripts/tasks/test-binary-smoke.sh (also wired into CI) asserts exact formatted output for TS and Go fixtures.
  • Dogfood: fmtkit format . over this repo itself produced the expected house-style formatting of the new files.

Adds the missing distribution piece that #46's module collapse set up:
one `fmtkit` binary per platform, published on GitHub Releases and
installable through the oullin/homebrew-fmtkit tap, with zero runtime
dependencies (no Node.js, no Docker).

- packages/devx/scripts/sidecar.ts: bun-compiled multiplexer that runs
  the TS pipeline, oxfmt, and oxlint from one executable, loading the
  napi bindings via NAPI_RS_NATIVE_LIBRARY_PATH.
- infra/scripts/release/stage-ts-assets.sh: builds the sidecar per
  platform and fetches the oxc-parser/oxfmt/oxlint bindings, pinned by
  packages/devx/package.json.
- packages/driver/internal/tsruntime: embeds the staged assets behind
  the fmtkit_sidecar build tag and extracts them to the user cache on
  first run.
- packages/driver/internal/orchestrator: Go port of the infra/bin/fmtkit
  pipeline with live-streamed, prettified tool logs plus the existing
  condensed summaries (--quiet restores summary-only output).
- packages/driver/cmd/fmtkit: the distributed entry point; subsumes the
  fmtkit-go surface under `fmtkit go`.
- .goreleaser.yaml + publish-release.yml `binaries` job: cross-platform
  tar.gz archives, checksums, and the Homebrew cask push (needs the
  HOMEBREW_TAP_TOKEN secret).
- tests.yml `binary` job: goreleaser config check plus an end-to-end
  smoke test of the compiled binary.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a self-contained fmtkit binary and release pipeline, embedding the TS toolchain (including the Bun-compiled sidecar and napi bindings) directly into the Go binary. It adds Go orchestrator logic to manage the pipeline execution and extract embedded assets on first run. The review feedback highlights opportunities to optimize memory usage by streaming embedded assets during hashing, reduce redundant string splitting in log summarization, and improve test robustness by quoting paths in shell scripts.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread packages/driver/internal/tsruntime/support.go
Comment thread packages/driver/internal/orchestrator/summarize.go Outdated
Comment thread packages/driver/internal/app/app_test.go Outdated
gocanto added 3 commits July 15, 2026 17:48
- orchestrator/cmd tests assert plain output; CI task runners export
  FORCE_COLOR, which injected ANSI codes into the captured buffers.
- setup-node v5 auto-detects pnpm from pnpm-lock.yaml for caching, but
  the binary jobs only use npm.
…tep flags

- move per-platform go:embed files to the module root so the staged
  bun sidecar + oxc binaries live in gitignored infra/bin/<os>_<arch>/
  directories instead of packages/driver/internal/tsruntime/embedded
- drop the now-empty tsruntime/embedded package; tsruntime reads the
  toolchain via fmtkit.SidecarAssets()
- add --ts / --go step-selection flags to format and format-all
  (no flags runs everything; unknown flags exit 2 with usage)
- cover project-local .oxlintrc.json overriding the bundled config
@gocanto gocanto marked this pull request as draft July 16, 2026 04:21
gocanto added 4 commits July 16, 2026 12:54
Addresses review feedback on #48:

- assetsDigest streams each asset through the hash via io.Copy instead of
  fs.ReadFile, so the 30MB+ bun sidecar is never held in memory.
- lastWithPrefix takes pre-split lines; summarizers split the log once
  rather than re-splitting per prefix lookup.
- Quote the log redirect target in the app test stub script so temp dirs
  containing spaces do not break it.
Bumps TypeScript 6.0.3 -> 7.0.2, @types/node 25 -> 26, oxc-parser
0.132 -> 0.140, oxfmt 0.41 -> 0.59, oxlint 1.66 -> 1.74, tsx 4.22 -> 4.23,
vite-plus 0.2.1 -> 0.2.4, viper 1.20 -> 1.21 and x/tools 0.43 -> 0.48.

Also promotes github.com/mattn/go-isatty to a direct require, matching its
existing use in the orchestrator logger.

Verified locally: go build, go test ./..., tsc --noEmit, devx check
(blank-lines/oxlint/oxfmt) and the devx test suite all pass. oxfmt 0.59
leaves the tree unchanged, so the embedded napi bindings move together
with the formatter that produced the current formatting.
oxfmt 0.59 and oxlint 1.74 lazily import("vite-plus") to read a Vite+
config file. bun compiles the sidecar from a throwaway workdir that
installs only oxfmt/oxlint/oxc-parser, so the bundler could not resolve
the specifier and the staging step failed:

    error: Could not resolve: "vite-plus"
        at node_modules/oxfmt/dist/cli.js:133:27

Upstream ships this specifier external on purpose so the user-installed
copy is used at runtime. Do the same, alongside the napi bindings and
prettier plugins already listed. The sidecar bundles .oxfmtrc.json and
.oxlintrc.json, so the Vite+ config path is unreachable there, and a
project that keeps its config in vite.config.ts carries its own
vite-plus for the import to resolve against.

Verified with infra/scripts/tasks/test-binary-smoke.sh on darwin_arm64.
@gocanto gocanto marked this pull request as ready for review July 16, 2026 05:58
@gocanto gocanto merged commit 7da8b3d into main Jul 16, 2026
20 checks passed
@gocanto gocanto deleted the feat/single-binary-distribution branch July 16, 2026 06:01
gocanto added a commit that referenced this pull request Jul 16, 2026
The self-contained binary (bun build --compile) hung forever on any file with
embedded code — .vue (<template>/<style>), markdown, HTML — while plain
.ts/.tsx were unaffected. Regression from #48 (the Docker→binary switch); the
Docker image formatted these fine.

Root cause: oxfmt formats embedded blocks through a Tinypool
`runtime: "child_process"` pool. The pool forks worker entry scripts
(oxfmt/dist/cli-worker.js, tinypool/dist/entry/process.js) that only exist as
virtual /$bunfs/root/ paths inside the compiled binary and are never carved out
as loadable files, so every worker dies on startup ("Worker exited
unexpectedly") and Tinypool.destroy() then awaits an exit event that never
fires — the process wedges until SIGKILL. Plain .ts never initialises the pool,
which is why it slipped through: the smoke fixture had no embedded-code file.

The four pooled functions (formatFile / formatEmbeddedCode / formatEmbeddedDoc /
sortTailwindClasses) are stateless prettier calls; the pool only bought
cross-file parallelism, not isolation. Rewrite oxfmt's worker-proxy at
release-staging time to call them directly on the main thread, dropping the
unusable worker layer. Behaviour is identical (the embedded-code fraction loses
parallelism but also sheds the per-worker prettier startup cost).

- infra/scripts/release/patch-oxfmt-inprocess.mjs: discovers oxfmt's hashed API
  module + export aliases, asserts the exact worker-proxy structure, and swaps
  the Tinypool block for in-process calls. Exits non-zero on any drift so an
  oxfmt bump can't silently reintroduce the hang.
- stage-ts-assets.sh: run the patch after npm install, before bun --compile
  (once; all four targets share the patched workdir).
- test-binary-smoke.sh: add a Vue SFC fixture exercising all three sections, so
  the embedded path is covered and a future hang fails the smoke test.

Verified: rebuilt the host binary; `fmtkit ts <sfc>.vue` now exits 0 in ~2s with
<script>/<template>/<style> all formatted; edgekit's 21 .vue files no longer
hang; all four release targets cross-compile; go test ./... and the smoke test
pass.
gocanto added a commit that referenced this pull request Jul 16, 2026
…vue (#51)

* fix: format embedded code in-process so the binary stops hanging on .vue

The self-contained binary (bun build --compile) hung forever on any file with
embedded code — .vue (<template>/<style>), markdown, HTML — while plain
.ts/.tsx were unaffected. Regression from #48 (the Docker→binary switch); the
Docker image formatted these fine.

Root cause: oxfmt formats embedded blocks through a Tinypool
`runtime: "child_process"` pool. The pool forks worker entry scripts
(oxfmt/dist/cli-worker.js, tinypool/dist/entry/process.js) that only exist as
virtual /$bunfs/root/ paths inside the compiled binary and are never carved out
as loadable files, so every worker dies on startup ("Worker exited
unexpectedly") and Tinypool.destroy() then awaits an exit event that never
fires — the process wedges until SIGKILL. Plain .ts never initialises the pool,
which is why it slipped through: the smoke fixture had no embedded-code file.

The four pooled functions (formatFile / formatEmbeddedCode / formatEmbeddedDoc /
sortTailwindClasses) are stateless prettier calls; the pool only bought
cross-file parallelism, not isolation. Rewrite oxfmt's worker-proxy at
release-staging time to call them directly on the main thread, dropping the
unusable worker layer. Behaviour is identical (the embedded-code fraction loses
parallelism but also sheds the per-worker prettier startup cost).

- infra/scripts/release/patch-oxfmt-inprocess.mjs: discovers oxfmt's hashed API
  module + export aliases, asserts the exact worker-proxy structure, and swaps
  the Tinypool block for in-process calls. Exits non-zero on any drift so an
  oxfmt bump can't silently reintroduce the hang.
- stage-ts-assets.sh: run the patch after npm install, before bun --compile
  (once; all four targets share the patched workdir).
- test-binary-smoke.sh: add a Vue SFC fixture exercising all three sections, so
  the embedded path is covered and a future hang fails the smoke test.

Verified: rebuilt the host binary; `fmtkit ts <sfc>.vue` now exits 0 in ~2s with
<script>/<template>/<style> all formatted; edgekit's 21 .vue files no longer
hang; all four release targets cross-compile; go test ./... and the smoke test
pass.

* refactor: split the oxfmt patcher into a TypeScript concern slice

Replaces the single patch-oxfmt-inprocess.mjs with a TypeScript concern slice,
per the repository TypeScript standards (class-based, organised by concern,
behaviour behind narrow interfaces, wired by constructor injection).

infra/scripts/release/oxfmt-inprocess/
  result.ts       Result vocabulary (stays functional — it is a value type)
  errors.ts       tagged errors, each carrying the anchor/path it looked for
  text-files.ts   TextFiles port + NodeTextFiles adapter (the only I/O)
  api-bindings.ts ApiBindings DTO: private ctor, static parse() -> Result,
                  readonly getters
  shim-source.ts  static-only renderer for the generated shim source
  cli-patcher.ts  OxfmtCliPatcher orchestrator: injected TextFiles, guards,
                  rewrite
  index.ts        barrel

patch-oxfmt-inprocess.ts is now a thin composition root: parse argv, construct
the adapter, call the service, report. Expected failures are values
(Result<PatchOutcome, OxfmtPatchError>) rather than process.exit() scattered
through the logic, so every drift in oxfmt's internals reports what it was
looking for.

Imports go through a #oxfmt-inprocess/* subpath map (new
infra/scripts/release/package.json) rather than ./ or ../ specifiers, mirroring
packages/devx/scripts/package.json. node runs the entrypoint directly via type
stripping, so staging still needs only node/npm/bun — no tsx.

Review feedback addressed: binding parsing now accepts a bare `localName` as
well as `exported as localName`, so a bundler that stops aliasing the API
re-exports no longer fails the patch.

Verified: tsc --strict (plus noUncheckedIndexedAccess,
exactOptionalPropertyTypes, noImplicitOverride, noFallthroughCasesInSwitch)
clean; `pnpm -C packages/devx check` (blank-lines + oxlint + oxfmt over all 59
tracked files) clean; go test ./... and test-binary-smoke.sh (including the Vue
fixture) pass; the staged binary still formats .vue in ~2s without hanging.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant